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CERTAIN PRINCIPLES OF BIOMORPHIC ROBOTS 

DESCRIPTION 

BACKGROUND OF THE INVENTION 

Field ©£ the Invention 

5 The present invention generally relates to 

methods of designing autonomous mobile robots and 
more specifically relates to the prediction of the 
sensory consequences of movement, learning 
affordances using neural methods, and exploitation 
10 of the natural system dynamics to simplify 

computation and robot control in autonomous robot 
explorers . 

Description of the Prior Art 

Technology has generally been aimed to make 
15 human life easier by taking on the burden of human 
tasks, or performing tasks that humans cannot 
perform due to physical constraints. In turn, 
robots have, and continue to be, developed that are 
mobile and that have the ability to retrieve or 
20 report information in accordance with this 

technological trend. In other words, robots are 
being designed to relate to humans while providing 
them with life simplifying solutions . To meet this 
goal, robots are taking on forms similar to either 
25 humans or animals, for purposes of cognitively and 

emotionally relating with the technology, as well as 
for patterning the evolutionary success in mobility 
of humans or animals (also hereinafter sometimes 
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collectively referred to as biol gical systems) . 
Additionally, a major reason for choosing a legged 
form, particularly a two-legged humanoid form, ia 
that humans have built a substantial environment 
S based on human mobility needs . as such, robots 

using wheels and/or tracks generally do not meet the 
mobility needs for a variety of terrains where 
legged robots are generally more successful. 

Bipedal locomotion over a flat, firm surface 
10 does not require visual or other type of distal 

sensory apparatus. However, if the environment is 
varied, vision or other type of distal sense is 
necessary to adjust gait in an anticipatory manner. 
Various visual cues are used by animals and humans 
15 to guide locomotion. These cues include cues that 
rely on or exploit the geometry of the environment: 
optic flow, stereopsis, depth from elevation and 
others as well as non-geometric cues such as the 
color, texture and surface patterns of the 
20 environment. 

Movement of an observer (biological or 
otherwise) gives rise to motion parallax with 
objects in the environment. Light reflected or 
emitted by surfaces in the environment give rise to 
25 a pattern of luminosity changes on the observer's 

retina or imaging surface. This pattern of changing 
luminosity is optic flow. Optic flow is highly 
correlated with motion parallax. Through the 
examination of the optic flow field it is possible 
30 to determine time to contact, and structure of the 

environment, and the movement of the observer, 
including direction and rotation. The latter 
phenomenon is sometimes referred to as visual 
kinethesis in the literature. Scientific studies 
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support the hypothesis that optic flow is essential 
for navigation of legged and flying biological 
systems in the environment. 

Additional geometric visual cues include 
5 stereopsis and depth from elevation. Stereopsis is 
used to determine visual sensory data about the 
environment of a biological system by comparing two 
or more images from slightly different view points, 
the arrangement of human eyes being the archetypal 
10 example. Stereopsis can convey information about 

the siae of an obstacle, although, in humans, It ie 
apparently less important than other modalities for 
judging distance to an obstacle and is not an 
essential sensory factor for locomotion. Depth from 
15 elevation is yet another visual cue which operates 
under the assumption that the observer is 
kinematically connected to the obstacle being 
observed. Thus, if the observer is connected to a 
plane, obstacles closer to the observer will appear 
20 lower in the visual field than obstacles further 

away. This simple effect is exploited in biological 
systems to judge distances to points in the 
environment. However, these geometric cues alone, 
although helpful, are not sufficient for advanced 

25 locomotion. 

Non-geometric visual cues mainly include 
texture, and color, but also encompass specular 
reflection or any other surface cue indicating the 
quality of a surface. These visual cues, when 

30 combined with geometric cues, can greatly enhance 
the success of locomotion as they assist the 
observer in anticipating surface characteristics- 
These visual cues aid biological systems in 
determining what characteristics a. surface may 
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xhibit, such as if a surface is slippery (e.g. 
lea) - 

The environment can » suggest' desirable foot 
placemenc for navigating a region. Figure 1 
5 illustrates a stone walkway partially covered by ice 
and snow; the highlighted gray regions indicate the 
more favorable locations for foot placement within a 
reasonable proximity to the path of intended motion- 
The suggestion for a particular foot placement and 
10 the motor action necessary to accomplish this action 
is called an affordanee. 

Affordanee encompasses how to perform an action 
but not the actual selection of such an action. The 
environment presents potential actions or 
15 affordances, and a choice is made as to which of the 
potential actions is the best pursuit. A person, 
seeing a mug, immediately perceives the many ways to 
grasp it, although there is no need for intermediate 
processing of *whaf the object is. Likewise, an 
20 animal, seeing a rock, immediately perceives a way 
to step over it, on top of it, or step around it 
depending upon the perceived size or shape of the 
given rock. Affordanee perception includes the 
motor capabilities of the observer. It is also 
25 largely linked to learning abilities, for example, 

if a choice was made to step over a rock that turned 
out to be too large to successfully maneuver over, 
and as a result the animal fell, the animal would 
learn not to try to step over the rock, and use an 
30 alternative approach instead. Past research has 

managed to link affordances to neural substrates in 
the brain. 

A key problem in the deployment of robots is 
that even the most agile robots, quadrupeds and 
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©sp cially bipeds, lack good affordance processes 
and can therefore be easily destabilized by 
obstacles. An affordance has the function of 
intelligent pattern matching: the current 
5 environment is matched to the set of possible motor 
actions that can be successfully executed by the 
animal or machine at a given time instant. This 
pattern matching can be quick and is superior in 
speed to methods that rely on algorithrrdcly driven 
10 geometric motion planning. 

Vision can assist in stabilizing the subject's 
relationship to the environment, as well as being 
essential for navigation, route adjustment and 
planning. Without vision, the situation is worsened 
15 as che robot moves faster and has less time for 

appropriate planning based on alternate sensory cues 
(e.g. tactile). It is desirable to replicate animal 
visual sensory ability in robots to learn 
affordances and react to the surrounding environment 
20 using the previously described geometric and non- 
geometric methods. A method for achieving this must 
be resolved for robots to euBuro successful mobility 
within a given environment. 

Currently, there is surprisingly little work on 
25 the tight integration of vision and locomotion. 

Historically, the two fields have been addressed by 
largely separate groups of researchers. 

Honda and Sony robots use vision for navigation 
(e.g. moving in the general direction of an 
30 obstacle) . The Honda Asimo bipedal robot— "biped" 

for short — walks on two legs and can maneuver up and 
down stairs, turn, and walk with a reasonable gaic. 
Sony has developed several generations of small 
quadruped robots called "Aibo", but has also 
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developed a biped robot, Sony's robots are viewed 
more as ^content delivery devices" which playback 
media concent developed by others, similar to a VCR, 
although exhibiting an appearance that is more human 
5 or animal in form. 

Robotics has become a field yielding many 
important application© for the U.S. Military as 
well. However, past declassified reports that 
tracked robotic vehicles being used in the field 

10 during search and rescue operations following the 
World Trade Center collapse lacked the required 
mobility to adequately perform in such applications. 
Legged robots were recommended following this report 
for increased mobility. 

15 As such, it is clear that there is a current 

and rapidly growing interest in legged robotic 
machines as well as a need for fast algorithm* to 
provide these legged robots with visuomotor 
coordination. 
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SUMMARY OF THE INVENTION 

It is therefore an object of the present 
invention to provide a robot that has the ability to 
simulate the integration of perception with action 
5 of biological systems thereby allowing the robot the 
capability of making visually triggered gait 
adjustments prior to and during stepping over an 
obstacle. 

It is an additional object o£ the present 

10 invention to provide a robot that has the ability to 
detect non-geometric stimuli, such as color/ 
texture , or other surface attributes and determine 
the utility functions resulting from such stimuli - 
In order no accoinplish these and other objects 

15 of the invention, a mobile apparatus is provided 

comprising at least one distal sensor for detecting 
an obstacle in at least the mobility path of said 
mobile apparatus and providing first data, at least 
one tactile or pressure sensor for determining the 

2 0 stability of the mobile apparatus providing second 
data, at least one active joint, and an algorithm 
for integrating perception in accordance with first 
data and/or second data with action of the joint (a) 
in performing a cyclic stride and/or adjustment of 

25 said cyclic stride to avoid an obstacle. 

Further , a method for determining gait 
adjustments in a mobile apparatus (robot) will be 
provided whereby said mobile apparatus may bypass an 
Obstacle. The method includes receiving raw visual 

30 data, determining what data within the raw data set 
is novel based on predictions , determining if an 
obstacle is in the mobility path of the mobile 
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apparatus by associating past patterns recorded by 
the mobile apparatus with past reflexes using a 
sensorimotor map, sending determined information to 
a central pattern generator (CP<3) to calculate and 
5 dictate motor commands and resultant movement of the 

mobile apparatus/ sending an error signal back to 
the sensorimotor map in the event that instability 
is detected by sensors on the mobile apparatus as a 
result of an obstacle, and learning to associate 

10 visual data with emerging obstacles in response to 
the destabilization of the robot in some way* 
Destabilization can be detected by analysis of 
signals from a tactile, pressure, or even a 
vestibular sensor, or even a joint sensor that 

15 senses a displacement or the limb from an expected 

trajectory. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing and other objects, aspects and 
advantages will be better understood from the 
following detailed description of a preferred 
5 embodiment of the invention with reference to the 
drawings/ in which: 

Figure 1 is a photograph of a walkway with 
obstacles, affordances for foot placement are 
highlighted. 

10 Figure 2 is a schematic view of the process 

relationships incorporated within the algorithm. 

Figure 3, comprising portions 3a , 3b, and 3c, 
is a graphical interpretation of the data used in 
the detection of novelty from the right visual field 
15 of an exemplary robot. 

Figure 4 is a schematic o£ the brain processes 
used when integrating perception with action. 

Figure 5 is a diagram illustrating the change 
in eye level depending upon the phase of gait, 
20 Figure 6 is a graphical representation of 

adaptive stride versus non-adaptive stride when 
approaching an obstacle. 

Figure 7 is an isometric view of a pseudo- 
cerebellum illustrating the sub-components and 
25 functions. 

Figures 8a and 8b are diagrams illustrating 
stumble correction reflex and stride correction. 

Figure 9a is a graphical representation of the 
weights in the sensorimotor trans format ion function 
30 for stride length adjustment, mapping perception to 

action, after learning has progressed. 

Figure 9b is a schematic representation of the 
structure of weights in Figure 9a- 
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Figure 10 is a schematic diagram of rh© process 
used to determine foot placement based on the 
surface characteristics of the environment (e.g. 
texture and color, highlights and any other surface 
5 cues . 

Figure 11a is a photograph of an exeinplary 
walkway with obstacles. 

Figure lib shows the area of the exemplary 
walJeway that presents desirable footfalls based on 
10 image segmentation * 

Figure 11c shows the areas of the exemplary 
wallcway that present obstacles based on image 
segmentation . 

Figure 12 is a diagram showing a typical test 
15 track with *good v surfaces for foot placement 
illustrated as circles. 

Figure 13 is an illustration of sparse 
footholds available for a robot to cross a gap. 
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DETAILED DESCRIPTION OF A PREFERRED 
EMBODIMENT OF THE ZNVENTION 

Referring now to the drawings and more 
particularly to Figure 2, a schematic view of the an 
5 exemplary mobile robotic biped 21 or "robot" having 

at least active hip joints 37 and "feet" 54 shown in 
relation to an exemplary obstacle 20. Figure 2 
visually demonstrates an algorithm in accordance 
with the invention that has been successfully 

10 employed in a robot 21. The algorithm is developed 

to model the same learning process and method that 
biological systems are believed to use for 
successful mobility. The algorithm consists of an 
autonomous system such as a central Pattern 

15 Generator 29 (e.g. a distributed system of non- 
linear limit cycle oscillators that generate the 
necessary pattern of control signals for limb 
movement), a pseudo-cerebellum 30 that is 
responsible for predicting sensory perception 39 and 

20 novel events 41 after receiving vision cues 35 from 
at least one distal sensor (e.g. a camera, multiple 
cameras/ laser rangefinder, etc.} 33 , and a system 
of "reflexes" 31, 32 that indicates the instability 
of the robot 21 for learning what constitutes an 

25 obstacle 20. This algorithm provides a method of 
associative learning between the pseudo-cerebellum 
3 0 and the autonomous system (e.g. central pattern 
generator 29) , propagating back through time, 
learning triggered by the % reflex system' comprised 

30 of sensors 31, 32, 33, thereby learning alternatives 
to actions, and ranks these actions (e.g. utility 
functions) to enhance prediction 39. Such utility 
functions can be provided by an expert in robotics, 
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by examination of human strategies, by analytical 
methods, or based on a learning algorithm such as 
reinforcement learning. 

The present invention focuses on creating 
5 affordance in robot© 21 to result in an ability to 
make visually triggered gait adjustments prior to 
and during stepping over a small obstacle 20. There 
are two key desirable behaviors in a robot 21 when 
surmounting an obstacle: (1) Foot placement 
10 adjustment and (2) stepping over the obstacle 20 at 
the correct time. 

The robot 21 faces a demanding perceptual 
problem in determining what constitutes an obstacle 
20 without being explicitly taught as both terrain 
15 with and without obstacles produce coirplex patterns 
of visual stimuli. In the present invention, an 
obstacle 20 becomes implicitly defined as any 
potentially destabilizing element of the 
environment, if the robot collides with the 
20 environment, it must refer back to the Sensorimotor 
Map 36 to determine what it saw previously and use 
that information (e.g. as seen in portion 3c of 
Figure 3) to adjust its control system not to make 
the same mistake again. 
2 5 Prior to the last steps before going over an 

obstacle 20, the robot must adjust its foot 
placement to step smoothly over the obstacle 20, 
Without these adjustments, the robot may need to 
break its stride. To avoid such an outcome, the 
30. robot must accurately predict a collision with the 
obstacle 20 and step at the correct time, and 
integrate the corresponding adjustment with the step 
cycle to prevent collision or loss of stable 
posture. This is called the step-over capability. 
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Foot-placement is extremely important in this 
process, and therefor©/ a method to provide accurate 
judgement for, and execution of, f oot-plaeement ie 
highly sought. 
5 Figure 4 illustrates a possible coordinating 

architecture for integrating perception with action 
in the neural system of biological systems . This 
schematic is based on studies of the cat and primate 
brain. Information acquired by the visual cortex 22 

10 is sent to the posterior parietal cortex 23 at which 
point r information is distributed through a variety 
of coordinating paths to the Central Pattern 
Generator (CP6) 24 , which ultimately results in 
movement in animals, although it is unclear if 

15 humans rely on a biological CPG 24 for locomotion. 
The spinal circuits (CPGs) 24 create the basic 
template for movement and send coordinating 
information to the cerebellum 25. The cerebellum 
combines sensory information and sends timing and 

20 possibly novelty information to the motor cortex 26 
by way of the thalamus 27. The motor cortex 26 then 
modulates the biological CPGs 24 creating a circular 
method of updating information and movement 
decisions. Information that is accjuired at the 

25 visual cortex 22 and is sent to the posterior 

parietal cortex 23 through the basilar pons 28 to 
the cerebellum 25 provides the necessary information 
to the CPG 24 to conpute actions and modifications 
thereof to accommodate features of the environment. 

30 It is important to recognize that information 
flowing between the biological CPG 24, the 
cerebellum 25 and the motor cortex 26 is used to 
establish coordination between modulator commands 
and the ongoing cycle of the CPG. The invention 
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provides an algorithm that artificially mulates how 
biological systems learn to step over obstacles. 

Referring now to Figure 3 comprising portions 
3a, 3b/ and 3c, a dynamic attention mechanism is 
5 shown that operates in a way such as to detect 

unexpected visual stimuli baaed on the state of all 
perceptual information and the locomotor controller 
(e.g. joint commands, tactile, disparity, and phase 
of gait information) . Figure 3 is accordingly 

10 divided into the three key layers of operation of 

the dynamic attention mechanism: raw data input 3a, 
prediction 3b, and novelty detection 3c - The 
process is demonstrated using data from the right 
side of the visual field only, but the procedure is 

15 identical for the left side as well. 

In the raw data layer shown in portion 3a of 
Figure 3, the activation of the right vector cells, 
with eighteen (18) elements, versus phase of gait 
(described as e in Figure 5), divided into twenty 

20 (20) discrete segments {each representing l/20th of 

a gait cycle) , for a total of three-hundred-and- 
sixty (360) cells 38. A gait cycle can be defined by 
assigning an arbitrary point as the beginning of the 
gait cycle. The unfolding trajectory until the 

25 beginning of the next gait cycle (reaching that same 
arbitrary point of motion) can be parameterized by a 
Single variable called phase. Cells 38 with lower 
numbers 3Ba, according to the graph, are closer to 
the robot whereas cells with higher numbers 38b are 

30 further away. The array of cells appears inclined 
consistent with a view of the surface from above at 
an oblique angle. Undulation in the phase direction 
corresponds to viewing height change during walking. 
Other more random variations thus represent 
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perceived (3a) or pr dieted (3b) surface 
irregularities or possible obstacles. 

in an exemplary prediction layer shown in 
portion 3b of Figure 3, the graphical representation 
5 of the predicted appearance of a surface is 

organised in cells 38 by disparity and phase as in 
portion 3a of Figure 3. Each cell 38 receives 
information about an area of the surface from all 
sensors 31, 32, 33 , (pref errably encoded in a sparse 

10 code) . The weight for each signal is determined by 
a learning rule (e.g. Widrow-Hoff LMS associative 
learning rule, etc.). Hie learning rule chosen is a 
supervised learning neural network learning rule 
although it may be possible to achieve the same 

15 results with an unsupervised learning rule as well. 

The primary function of the learning rule is to 
change the input weights of each cell such that it 
becomes a better predictor of sensor stimuli as time 
progresses. The learning rule reduces the weight 

20 from sensors with little predictive value and 

increases those with greater predictive value. Tlie 
prediction is generated by a weighted average of all 
sensory and motor data (phase, motor signals 
(efference copies) tactile sensation, etc.) This 

25 adaptation is continuous through the 'life' of the 
robot 21 as the overall architecture is robust 
against loss of any sensor modality as all sensory 
information contributes to prediction of each other 
sensor. 

30 * s arranged in Figure 3, an exemplary novelty 

layer 3c, receives the difference between the raw 
data layer 3a and the prediction layer 3b weighted 
by a variable gain factor in order to determine an 
obstacle without being explicitly taught. The gain 
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factor 42 for novelty detection varies due to a 
local feedback mechanism. The gain adjusts to 
maintain a low average activity at all times. If a 
certain cell has little predictive value, the cell's 
5 oain is reduced. if other cells predict the actual 

sensory input very accurately, that cell's gain is 
increased, allowing finer discrimination. The 
output function of the novelty layer 41 represents a 
hard- limit threshold. 

1° Thus, the dynamic attention mechanism, shown in 

Figure 3, allows the robot 21 to detect fine 
environmental features (e.g. an obstacle 20) of 1 cm 
in height or less whereas without the predictive 
component of this mechanism, the otherwise same 

15 device could not reliably detect obstacles less than 
5 cm in height. As such, even small disparities 
between the actual /perceived (e.g. raw data layer 
40) and predicted features (e.g. prediction layer 
33) , that correspond to just a fraction of a 

20 disparity value are recognized as novelty. 

Disparity has been defined by those well versed in 
the field of binocular vision and stereopsis as the 
side to side (horizontal) or up and down (vertical) 
"difference in the position of similar images in the 

25 two eyes... and can produce a compelling sensation of 

three-dimensionality." In this implementation, 
disparity values can easily vary +/-1 disparity 
value for a particular cell during walking and 3-4 
disparity values between cells. Learning converges 

30 quite rapidly using this method such Chat good 

predictions and expectancy are obtained within one- 
hundred-and- twenty (120) seconds after initiation. 

More particularly, Figure 7 illustrates the 
pseudo-cerebellum 30 in which the dynamic attention 
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mechanism (Figure 3) functions. The pseudo- 
cerebellum 30 reacts to the information derived from 
the distal sensor (s) 33 to perform dynamic attention 
mechanism functions in each of the subregions 43 of 
5 the pseudo-cerebellum. Each subregion predicts 

sensory information based on both visually geometric 
stimuli including optic flow and other distal cues, 
as well as tactile stimuli and vestibular stimuli. 
The stimuli of each subregion 43 is in terms of 
10 distance (e.g. near stimuli to far stimuli). Within 

each subregion 43, prediction 39 is made in 
consideration of an eff erence copy 44 and other 
sensory input 31,32 using the formula 

f ( (x • w)-t) 

15 where f is the neural output, x is a vector of 

inputs, sparsely coded, w is a vector of weights, 
and t is a threshold value. The function must have 
a non-linear form and can be as simple as f (x) = 
max(0,x), a sigmoidal function or a tanh(x) 

20 function. 

The prediction 39 is then compared to the 
actual visual elements where these elements are 
subtracted from the prediction layer 39 with the 
results of this difference being reevaluated with 

25 the prediction layer 39 and analyzed using adaptive 
gain 42 in order to determine novelty 41. The 
adaptive gain works as follows: at each step, two 
variables are accumulated. One variable indicates 
the number of times the cell has been active. The 

30 other indicates the number of potential times the 
cell could have been active. The ratio of the two 
indicates the fraction of times the cell has fired. 
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If this fraction is above a target value, say 0.05 
(or 5%) , chen the threshold for firing is raised by 
a small increment. If it is below this target: 
threshold, the cell threshold is reduced. In this 
5 way, a constant average firing rate is maintained. 

After a brief learning period, the robot 21 can 
accurately predict novelty 41 based on afferent 
responses. As illustrated in Figure 2, the 
information collected by the pseudo-cerebellum 30 

10 using the dynamic attention mechanism (Figure 3) is 
then processed for association using a Sensory Motor 
Map 36, which can be modified later in the event an 
error occurs. The sensory data that is referenced 
as having resulted in that error will then be 

15 recognized and avoided to refrain from future 
repetition of that error. As a result of the 
robot's learning capabilities and expectancy, the 
robot 21 also learns to expect a smooth surface in 
front of it when trained on a smooth surface, and 

20 without being explicitly told about smooth surfaces 
(or a rough surface whan trained on a rough surface, 
without being explicitly taught about rough 
surfaces) . 

The same algorithm has been applied 

25 particularly to tactile foot prediction and also to 
vestibular data (e.g. as the ear functions as an 
organ of balance), using foot pressure sensor (s) 32, 
By using the same techniques for novelty detection 
as in the case of visual input, the robot can easily 

30 detect an experimenter's light touch or other subtle 
disturbances (including angular and tranalational 
acceleration) during locomotion through the pressure 
sensor (s) 32. 

Vestibular data gives translational and angular 
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accelerations. In animals, angular accel ration is 
sensed by the s«mi«circular canals while 
trans lational acceleration is sensed by the otolith 
organs- Likewise, their man made counterparts 
5 (translational and angular aecelerometers) can sense 

angular and translational accelerations, and 
gyroscopes can sense angular velocity. 

Finally, based on the information determined in 
the pseudo-cerebellum 30 and confirmed in the 

10 Sensorimotor Map 36, as shown in Figure 2, the CPG 
29 can determine motor commands for the hips 37 to 
carry out through fixed rotation (e,g, walking); the 
details of which are not important to the basic 
principles of the invention- The CPG 29 discussed in 

15 Figure 2 is different from the CPG 24 discussed in 
Figure 4 as CPG 29 is artificial and not biological 
as CPG 24. The biological term has been applied to 
the artificial CPG 29 for the purpose of this 
invention as the two are based on the same key idea: 

20 that there is a system with a preferred 

implementation as a distributed system of non-linear 
oscillators, that can be modulated so aa to achieve 
more than one gait pattern or modulation (or 
modification) of one or more gait pattern. However, 

25 regardless of the actual implementation details, 
this locomotor controller must generate a signal 
indicating the Gait Phase in order that the pseudo- 
cerebellum 30 function in its assigned role. 

If , while walking, the tactile sensors 31 or 

30 pressure sensors 32 detect an error, the error is 
routed back to the Sensorimotor Map 36 in order to 
associate the previous actions and sensor inputs 
leading up to the error with the given outcome. 
Through this learning method, the robot 21 will 
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learn how to avoid repeating this error in the 
future , 

Stride length can be adjusted during locomotion 
by arranging the CPG 29, learning modules (e.g. 
5 prediction 39, and sensorimotor map 36), visual 
perceptual modules 33/ and tactile and pressure 
reflexes 31 and 32 in algorithmic combination. 
Figure 6 exhibits how minor changes can be made to 
the gait in order to keep stride while successfully 

10 stepping over an obstacle (e.g. adaptive stride 52), 
compared to how the robot would collide with the 
obstacle if gait adjustments are not made (e-g* non- 
adaptive stride 53). 

The robot 21 can learn to adjust stride length 

15 based on an activated novelty cell (e.g. something 

other than predicted) trigger* an eligibility trace - 
An eligibility trace is a short-term memory delay 
signal which allows association between future and 
current events; if the robot's foot collides with 

20 the environment, a training signal 45 (representing 
error) is sent to a sensorimotor mapping mechanism 
36 from the novelty cells (shown in portion 3c of 
Figure 3) to a variable that adjusts stride length 
in the CPG 29 . 

25 The response to the training signal 45 can be 

positive o+ (increase stride length) or negative 5- 
( decrease stride length) . The actual amount of 
weight adjustment in the positive or negative 
direction is a function of the training signal and 

30 the eligibility trace. When the training signal is 
triggered, the resultant stumbling of the robot 
creates * stumble correction." During a stumble 
correction reflex, the foot is first brought 
backward away from the obstacle, and then elevated 
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to avoid collision with the obstacle- As shown in 
Figures 8a if the foot collides with the obstacle on 
the way up, two inference can be made. First, it is 
likely that the foot should have been placed 
5 slightly back from the obstacle. Therefore, the 

learning algorithm adjusts the sensorimotor map to 
Shorten strides upon encountering a similar obstacle 
in the future (although, when the collision has 
already occurred, the robot will lengthen its stride 

10 in this situation to bypass the obstacle) . Second, 
the robot should have elevated its foot further than 
it did. Likewise, in Figure 8b, if the foot 
collides with an obstacle on the way down, the 
stride is adjusted so as to completely clear the 

15 obstacle and similar obstacles in the future and 
avoid stepping on the obstacle or a similar 
obstacle. At the time that the stumble correction 
is activated in the scenario of Figure 8b, the 
stride is prematurely terminated (shortened) to step 

20 onto the obstacle, however the algorithm for 
learning will lengthen the stride in future 
encounters so as to completely clear the obstacle 
20, in contrast to the reaction of the stumble 
correction reflex, in sum, the occurrence of the 

25 stumble correction reflex when the foot is on its 

way up (Figure 8a) or a collision with the obstacle 
20 on the foot's way down (Figure 8b) will cause a 
modification of two sensorimotor maps. One map is 
to adjust the stride length and the other is to 

30 trigger a step over response upon future encounters 
with similar stimuli. No error feedback is triggered 
if the robot steps onto the obstacle without 
becoming unstable. 

The purpose of this algorithm is to determine 
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correlations (e.g, maps) between visual input and 
modulation of the GPG 29 in the same manner as a 
biological system might process such information so 
that when an obstacle is detected at a distance 
5 along the intended path the robot 21 will gradually 
adjust its stride length prior to encountering the 
obstacle 20 in order to be able to step at a 
sufficient height at the correct time from a 
suitable location. When practiced correctly, the 

10 robot will be able to bypass the obstacle without 
hesitation or interference. This elegant stepping 
solution captures key points of biological processes 
including the spinal /cerebellar/ cortical loop, 
continuous learning throughout life, and direct and 

15 efficient mapping between the stages of perception 

and action. The algorithm can to lead to verifiable 
predictions in biological and human systems. 
Currently, evidence exists that humans decrease 
footfall variance upon approach to an obstacle, yet 

20 footfall variance in relation to obstacle height has 
not yet been confirmed although there is evidence 
that it will be in the future- As such, when the 
algorithm is implemented in the pseudo-cerebellum 
and cycled through the Sensorimotor map, foot 

25 placement becomes more tightly controlled the robot 
gets closer to the obstacle, and with increase of 
obstacle height. 

After learning a sufficient amount about the 
environment to accurately determine obstacles in the 

30 robot's intended path/ a pattern of weights that map 
the novelty cells to modulation of the locomotor CPO 
29 is depicted in Figure 9a, They appear as 
interleaving bands 46 of positive and negative 
weights simplified schematically in Figure 9b- 
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Depending on the band that the obstacle ppears in, 
the robot will determine whether it must shorten its 
stride or lengthen its stride. 

The pattern of the weights is reminiscent of 
5 spat io temporal filters for velocity estimation in a 
1-D array. However, while the cells are responsive 
to moving objects, speed is not measured as distance 
per unit time, but rather , distance per unit phase. 
Perception 39 is thus scaled to the size of the 

10 robot 21. Interestingly, there is no need for 

calibration of the sensor or motor apparatus for 
this system to work. The sensorimotor map is 
developed ab initio, without this information, as 
would happen in a biological system. 

15 The algorithm of the present invention has 

particular advantages over other systems. First, it: 
is computationally efficient. The algorithm can be 
placed in congpact customized neuromorphic chips for 
extremely fast, low power and extremely inexpensive 

20 operation. Second, this algorithm learns "what is 
an obstacle" for a particular robot, automatically 
adapting to the capabilities of the given platform 
in regard to its stability or instability. Third, 
the system can automatically compensate for the up 

25 and down movement (or any other periodic movement ) 

of the robot without the need for an image 
stabilization device. Finally, the system is 
applicable to any biped and can be extended to 
robots with a fewer (e.g. monoped hoppers) or 

30 greater (e.g. quadruped, etc.) number of legs. 

Additionally, as previously discussed, optic 
flow is the dominant visual sensor and is necessary 
for successful locomotion in biological systems* 
However, optic flow has not been used in the past to 
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d tect obstacles during legged locomotion. The 
previously described embodiment manages successful 
locomotion without using optic flow, whereas an 
alternate embodiment, described below, incorporates 
5 this iLmportant sensory technique. In. this alternate 

embodiment, sheer of normal flow field is the 
perceptual cue, and the robot should halt when 
aberration in the flow field is detected. In this 
process, prediction 39 is an optional step within 
10 Figure 7. In experiments, the robots were much more 
successful in detecting objects of small height 
(down to 1 cm) using prediction 39, whereas robots 
not using prediction were fairly comparable in 
detecting objects having heights of 4 cm or greater 
15 but could not recognize objects smaller than 4 cm in 
height with any reliability. Overall, using optic 
flow, when combined with prior methods, either with 
or without prediction, can be successfully used to 
control locomotion. 
20 The benefits of using geometric sensory data, 

including optic flow in real robots can be great. 
The prediction of sensory consequences of movement 
makes the system much more sensitive to fine 
features and novelty detection generally makes 
25 learning more efficient- The process of 

incorporating and updating feedback can shape the 
'perception' to the motor ability of the observer. 

A perfecting feature or alternate embodiment of 
the present invention may use geometrical or non- 
30 geometrical cues alone or in combination and non- 
geometric cues may be processed either with or 
without learning. In the dynamic fusion model as 
illustrated in Figure 10, representing this 
alternate embodiment or perfecting feature, the 



tNov.1 4. 2003 1:53PM Enterprise Works 



t-qNo . 0632 J/04i P - 26 26 



082000Q2AA 

25 

robot is very sensitive to non-geometric information 
(e.g. textures or surfaces) . The robot takes in an 
initial image of the textured environment 47 , 
recognizing a variety of textures (e.g. by any known 
5 feature recognition or extraction technique, and 
separates these various textures into distinct 
surfaces 4e, determining all of the areas exhibiting 
each texture separately. The robot can then 
determine a utility function 49 for each of the 

10 textured groups (e.g. stable, slippery, etc.) The 

robot will then recorabine the textured surfaces into 
an image based on the utility functions 49 and their 
usability. Based on this information, the robot 21 
can determine foot placement targets 50 and then 

15 activate the stride command to navigate the area. 

This concept is further illustrated in Figure 
11a, lib, and lie, where the robot takes in an 
initial image Figure 11a, recognizes the desirable 
fool-falls present in Figure 11a and isolates them 

20 in Figure lib (e.g. stones), as well as the 

isolating the obstacles in Figure lie (e.g. snow). 
This method can be carried out on a typical test 
track as shown in Figure 12 where the circles or 
targets 50 represent *good" surfaces for foot 

25 placement, where the targets are arranged at non- 
regular intervals. In tests, robots were able to 
utilize the targets 50 with great accuracy, with all 
errors being attributed to mechanical errors in the 
robots, and not with the technique itself. This 

30 technique can also be used to walk across a gap 

using sparse footholds 51 as shown in Figure 13 - 

Benefits o£ using non-geometric sensory data 
are very important to the improvement of robotic 
mobility. Using non-geometric sensory data, a robot 
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can incorporate n-poseible regions (as in a real 
biological system) as well as predicting bifurcation 
in motor performance. Additionally, the robot can 
support dynamic incorporation of surfaces and 
S obstacles into utility functions. For exan^le, 

consider a robot on a smooth surface with a small 
gap of ice between the robot and a firm foot hold. 
In this case the robot will step over the ice. If 
the gap of ice between the robot and the next solid 

10 foothold were to increase to a recognizably critical 
width, the robot / using the algorithm, will suddenly 
choose to make short careful steps on the ice rather 
than a long step to the solid surface. At this 
crtical point, the robot judges a small step on the 

15 ice to be as risky, or more so, as making an 
exaggerated long step to a aolid surface. 

The particular manner in which geometrical and 
non-geometrical cues are combined is not, itself, 
important to the basic principle of the invention in 

20 allowing a robot to accommodate particular obstacles 
or other features of the environment. However, it 
is generally the case that geometrical cues are 
principally processed for distinguishing and 
avoiding potentially destabilizing features of the 

25 environment (e.g. unsuitable foot placement 

locations) in an anticipatory manner which maintains 
the efficiency of gait while non-geometrical 
processing has particular utility in determining 
suitable locations for foot placement, generally in 

30 substantially real time for individual steps. 

Therefore, the availability of use of both types of 
cues, as provided by the invention, provides a 
powerful tool for enhancing robotic airtoulatory 
performance in a wide variety of environments. 
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In view of the foregoing, the described 
embodiments have the capability to visually adjust 
movement in a computationally efficient manner; 
determining where obstacles lie, and learning how to 
5 form af fordances to respond to the obstacles 

effectively without requiring explicit teaching of 
the obstacle or the environment. 

While the invention has been described in terms 
of a single preferred embodiment, those skilled in 
10 the art will recognize that the invention can be 
practiced with modification within the spirit and 
scope of the disclosed invention- 



